On the effectiveness of Gated Echo State Networks for data exhibiting long-term dependencies

In the context of recurrent neural networks, gated architectures such as GRU have contributed to development highly accurate machine learning models that can tackle long-term dependencies in data. However, training networks is performed by expensive algorithm gradient descent with backpropagation through time. On other hand, reservoir computing approaches Echo State Networks (ESNs) produce be trained efficiently thanks use fixed random parameters, but are not ideal for dealing data presenting dependencies. We explore problem employing ESNs from both theoretical and empirical perspectives. do so deriving evaluating a necessary condition non-contractivity state transition function, which important overcome fading-memory characterization conventional ESNs. find using pure methodologies sufficient effective gating mechanisms, while instead even only gates terms predictive accuracy.